The present invention relates to feature extraction and identification in medical imagery. More particularly, the invention relates to methods for generating efficient models of complex anatomical structures in the presence of xe2x80x9cleakage,xe2x80x9d and methods for employing such models as an aid to diagnosis.
The science of medical image processing has taken tremendous strides in the past two decades, particularly in the field of three-dimensional visualization of internal anatomical structures. Such three dimensional models can be virtually rotated and viewed from any perspective, providing invaluable insights to surgeons, diagnosticians, researchers, and other scientists.
In its raw form, medical imagery typically consists of a large array of numbers representing the value of a physical property (e.g. radiological xe2x80x9cdensityxe2x80x9d or xe2x80x9cintensityxe2x80x9d) at each of a plurality of regularly-spaced locations within the patient. The methods for generating structural models from this data proceed by generally well known principles.
One familiar class of techniques is known as xe2x80x9cvolume growingxe2x80x9d (sometimes termed xe2x80x9cregion growingxe2x80x9d). In accordance with these techniques, a seed voxel (volume element) is first identified within the anatomical structure of interest. Other voxels are successively analyzed and identified as belonging to the same structure if (1) they adjoin a voxel already identified as belonging to the structure, and (2) they meet a specified physical attribute (typically a radiological density in a range characteristic of the structure of interest).
According to standard region growing methods, after the seed voxel is identified, the six voxels sharing a face with the seed voxel are analyzed to determine if their physical attribute is within the specified range. If so, such voxels are marked as belonging to the structure. These voxels form a first tier of volume growth.
Next, each voxel in the first tier of volume growth is processed like a seed voxel, with adjoining voxels analyzed to determine whether their physical attributes are within the specified range. Voxels so identified form a second tier of volume growth.
This process continues, each iteration adding a shell of further voxels within the structure of interest.
In the simple case, this march of voxel cubes proceeds until growth in each direction is stopped by an exhaustion of voxels meeting the specified physical criterion. Collectively, the set of voxels thus identified fills the volume of the anatomical structure being analyzed, permitting its three-dimensional modeling.
The below-cited General Electric patents more fully detail the foregoing volume growing techniques and improvements thereto, including techniques for particularly locating the structure""s bounding surface with reference to the vertices of the outermost voxels, techniques for smoothing/shading the bounding surface to facilitate viewing, etc.
A problem with foregoing technique, and most other volume growing algorithms, is that of xe2x80x9cleakage.xe2x80x9d Leakage occurs when the march of cubes proceeds through the boundary of the structure being analyzed, rather than stopping as intended. Leakage causes the region growing to continue on the other side of the boundary, with a large number of voxels on the other side of the boundary spuriously identified as belonging to the structure of interest.
Leakage can occur for many reasons, including voxel dimensions larger than the boundary thickness, noise-induced imperfections in the bounding surface image data, etc.
As a general matter, leakage does not seriously impair the clinical usefulness of the extracted model. The leakage is an aesthetic distraction, but a reviewing physician can usually readily identify the leakage as a computer processing glitch. A more serious problem is the additional processing burden that leakage imposes on the three dimensional modeling software.
The computational complexity of three dimensional modeling is substantial. Such models typically include hundreds of thousands of data points, each of which must be processed every time the displayed model undergoes any change.
Much of the value of three dimensional modeling comes from the physician""s ability to rotate the model, move his point of perspective, and zoom into and out from features of interestxe2x80x94all in real time. Each such operation requires that the display screen be xe2x80x9crepaintedxe2x80x9d several times in quick succession to avoid the impression of jerky movement. Each such screen redraw, in turn, requires an enormous number of computations. The problem with leakage is that it vastly swells the dataset that must be processed, slowing the modeling software response time, and interfering With the physician""s sense of real time interactivity.
The leakage problem in region/volume growing algorithms has been recognized for decades, and has been dealt with in various ways.
One way is simply to adopt a feature extraction technique relatively immune to leakage. One such class of techniques relies on deformable models. In xe2x80x9cA Novel Volumetric Feature Extraction Technique, With Applications to MR Images,xe2x80x9d Int""l. Conf. on Image Proc., pp. 564-67, IEEE (1995), for example, Ashton et al model an expanding bubble whose expansion continues until the bubble fills the structure of interest. The shape of the bubble is controlled by a constraining force imposed by surrounding tissue and by a penalty for deviation from the expected surface normal. More particularly, Ashton et al. expand a seed voxel outwardly in an ovoid shape until the expected volume is reached, or until no further expansion is possible due to constraining tissue. (The ovoid shape is tailored in accordance with a priori information about the expected shape and size of the structure of interest.)
Leakage is rarely an issue in such deformable model techniques because, like a balloon, the expanding outer surface will not generally tunnel through a small opening and spawn a large ballooned volume on the other side.
Feature extraction techniques offering immunity to leakage are rare, and suffer from various drawbacks that have prevented their widespread adoption. Accordingly, various other solutions to the leakage problem have been proposed.
One solution has keyed on the characteristic shape of leakage volumes (i.e. a growth volume linked to a more central volume by a single (or a few) voxels). Computerized feature recognition techniques can be applied to identify such characteristic shapes and automatically delete them from the dataset. However, such approaches are generally disfavored in medicine due to the possible inadvertent deletion of clinically significant features.
In cases where the boundary of concern is thin, leakage can sometimes be ameliorated by employing commensurately small voxels (or subvoxels). With this approach, a boundary won""t be missed by a voxel simply spanning the physical boundary. However, halving the edge size of the voxel effects an eight-fold increase in the number of voxels to be processed, with a corresponding slow-down in manipulation of the resulting three-dimensional model. Moreover, thin boundaries are sometimes simply not manifested in the image data being analyzed, due to inherent resolution limitations of the data acquisition device (e.g. CT or MRI scanner). In such cases, small voxels offer no solution.
U.S. application Ser. No. 07/797,893, cited in U.S. Pat. No. 5,553,207 to Sekiguchi et al, proposes an interactive solution to the leakage problem in which an operator monitors progress of the volume growth on a plurality of display devices and interrupts the process if a leak extends to a volume outside the structure of interest. (Several display devices are required due to growth in three dimensions.) Sekiguchi""s ""207 Patent extends this technique by facilitating deletion of the spurious growth by reverse expansion from an operator-identified voxel within the leaked volume.
A drawback of Sekiguchi""s technique is its requirement of human interaction and real-time vigilance, increasing the cost of the diagnostic imagery and reducing the clinician""s productivity. Another drawback is its failing in the context of complex branching structures. In such structures, a leak may not be manifested as a swelling blobxe2x80x94readily apparent to an operator, but as a tunneling path that snakes and grows in a chaotically-bound volume outside the structure of interest. Such spurious growth may not be obvious to a monitoring operator, but nonetheless swells the dataset that is later manipulated for three dimensional display to a reviewing physician.
Another approach to the leakage problem is to define a boundary (e.g. a parallelepiped) beyond which volume growth is not permitted. Each time growth to a new voxel is considered, its x, y and z coordinates are checked to insure that each is within the prescribed limits.
While the foregoing boundary constraint avoids unbounded leakage, significant leakage can still occur, resulting in awkward delays in manipulation of the three-dimensional model.
U.S. Pat. No. 4,905,148 to Crawford considers and dismisses several approaches to the leakage problem, including (a) manually identifying potential bridges before the connectivity algorithm is applied; (b) circumscribing the stricture of interest with a user-defined boundary; and (c) requiring a higher order of connectivity (e.g. several approaches generally suitable, Crawford instead proposes a technique employing several seeds: one inside the structure of interest, and one or more outside. Region growing is first applied to the outside seed(s) using a directional criteria chosen to avoid the structure of interest (e.g. by specifying growth only in directions away from the structure of interest). The values of voxels identified by this first operation are then modified. Region growing can then proceed from the first seed voxel without possibility of leakage to those voxels whose values were modified.
Crawford""s method is illustrated in the context of preventing region growth in a skull from leaking outside through, e.g., eye socket cavities. In this context the area of potential leakage is large and obvious: the volume outside the skull. In many other contexts, however, this is not the case. Many anatomical structures have complex branching topologies, preventing a clinician from readily identifying regions of likely leakage. Moreover, Crawford""s technique requires operator-assisted pre-processing of the data, a step which is costly and often impractical.
In accordance with one aspect of the present invention, the foregoing and other drawbacks of the prior art are overcome. Leakage outside a structure of interest is constrained by an operator-set distance parameter. Unlike the simple bounding volume constraint of the prior art, this distance parameter refers to the tortuous path length actually traversed by a branch of a growing volume, rather than a straight-line distance between the beginning and end points. (A conventional bounding volume can be imposed as a secondary, fail-safe constraint, in case the anatomy outside the structure does not impose a chaotic path on the leakage volume.) The relative growth rates in three dimensions can be specified independently to provide for more efficient extraction of features whose shapes are generally known.
By the foregoing arrangement, leakage in complex anatomical structures is controlled without human intervention, yielding models better suited for rapid three dimensional manipulation.
One type of rapid three dimensional manipulation that would be unthinkable if a model were burdened with the large leakage volumes, but which is practical if such volumes are controlled, is virtual navigation of the model. In such methods, a physician xe2x80x9csteersxe2x80x9d himself through the structure using a joystick or the like, with an associated display being updatedxe2x80x94seemingly in real-timexe2x80x94in accordance with the joystick""s movements. This arrangement allows a physician to conduct a virtual tour of an anatomical structure, visiting features of interest while ignoring others.
While a random virtual exploration of the anatomical structure can be informative, better use of the physician""s time may be made by use of a guide. In accordance with a further aspect of the invention, a compilation of sites of potential interest is generated by a computer analysis of the imagery data, and serves to guide the physician in his virtual review of the anatomy. In one embodiment, this guide data is employed to direct the virtual tour, automatically navigating through the structure until a feature of interest is centered in view, and then pausing. The physician can then inspect the feature, using the joystick to move around as desired. After the physician has taken whatever note of the feature is merited, the tour is resumed, with the system navigating the physician to the next feature, and so on.
In another embodiment, the guide does not automatically navigate from virtual location to location for the physician. Instead, the guide data is used to highlight features of interest, e.g. by changing their color, so that the physician can take note of them on a self-guided tour.
In both these embodiments, a second 2D map-like display can be employed to identify the position of the physician""s virtual perspective within the model, so as to avoid disorientation.
In an illustrative embodiment, the discernment of features for inclusion in the guide is performed by reference to their shape. In an embodiment tailored for diagnosis of bronchial pathologies, for example, functions of partial derivatives of a parameterized surface, and functions of a surface displacement vector, are employed. In another embodiment, a 3D filter computes partial derivatives of a surface model. The 3D filter visits each vertex in the model and filters neighboring voxels to compute the partial derivatives of the surface at the vertex. In both embodiments, the partial derivatives are used to compute curvature characteristics. By comparing the curvature characteristics with predetermined characteristics, surface anomalies can be detected and highlighted for review. These methods have been found to accurately characterize the shape of polypoid lesions found in the bronchus and can be adapted for other anatomical structures as well.
By the foregoing arrangement, a physician""s attention is advantageously focused on features of potential clinical significance, enhancing the physician""s effectiveness and improving patient care.
The foregoing and additional features and advantages of the invention will be more readily apparent from the following detailed description, which proceeds with reference to the accompanying drawings.